In this project, you'll define and train a DCGAN on a dataset of faces. Your goal is to get a generator network to generate new images of faces that look as realistic as possible!
The project will be broken down into a series of tasks from loading in data to defining and training adversarial networks. At the end of the notebook, you'll be able to visualize the results of your trained Generator to see how it performs; your generated samples should look like fairly realistic faces with small amounts of noise.
You'll be using the CelebFaces Attributes Dataset (CelebA) to train your adversarial networks.
This dataset is more complex than the number datasets (like MNIST or SVHN) you've been working with, and so, you should prepare to define deeper networks and train them for a longer time to get good results. It is suggested that you utilize a GPU for training.
Since the project's main focus is on building the GANs, we've done some of the pre-processing for you. Each of the CelebA images has been cropped to remove parts of the image that don't include a face, then resized down to 64x64x3 NumPy images. Some sample data is show below.

If you are working locally, you can download this data by clicking here
This is a zip file that you'll need to extract in the home directory of this notebook for further loading and processing. After extracting the data, you should be left with a directory of data processed_celeba_small/
# can comment out after executing
#!unzip processed_celeba_small.zip
'unzip' is not recognized as an internal or external command, operable program or batch file.
data_dir = 'processed_celeba_small/'
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import pickle as pkl
import matplotlib.pyplot as plt
import numpy as np
import problem_unittests as tests
#import helper
%matplotlib inline
The CelebA dataset contains over 200,000 celebrity images with annotations. Since you're going to be generating faces, you won't need the annotations, you'll only need the images. Note that these are color images with 3 color channels (RGB)#RGB_Images) each.
Since the project's main focus is on building the GANs, we've done some of the pre-processing for you. Each of the CelebA images has been cropped to remove parts of the image that don't include a face, then resized down to 64x64x3 NumPy images. This pre-processed dataset is a smaller subset of the very large CelebA data.
There are a few other steps that you'll need to transform this data and create a DataLoader.
get_dataloader function, such that it satisfies these requirements:¶image_size x image_size in the x and y dimension.To create a dataset given a directory of images, it's recommended that you use PyTorch's ImageFolder wrapper, with a root directory processed_celeba_small/ and data transformation passed in.
# necessary imports
import torch
from torchvision import datasets
from torchvision import transforms
def get_dataloader(batch_size, image_size, data_dir='processed_celeba_small/'):
"""
Batch the neural network data using DataLoader
:param batch_size: The size of each batch; the number of images in a batch
:param img_size: The square size of the image data (x, y)
:param data_dir: Directory where image data is located
:return: DataLoader with batched data
"""
# TODO: Implement function and return a dataloader
transform_train = transforms.Compose([transforms.Resize(image_size),
transforms.ToTensor()])
image_dataset = datasets.ImageFolder(root=data_dir, transform=transform_train)
n_workers = 0
train_loader = torch.utils.data.DataLoader(dataset=image_dataset,
batch_size=batch_size,
shuffle=True,
num_workers=n_workers)
return train_loader
celeba_train_loader with appropriate hyperparameters.¶Call the above function and create a dataloader to view images.
batch_size parameterimage_size must be 32. Resizing the data to a smaller size will make for faster training, while still creating convincing images of faces!# Define function hyperparameters
batch_size = 32
img_size = 32
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# Call your function and get a dataloader
celeba_train_loader = get_dataloader(batch_size, img_size)
Next, you can view some images! You should seen square images of somewhat-centered faces.
Note: You'll need to convert the Tensor images into a NumPy type and transpose the dimensions to correctly display an image, suggested imshow code is below, but it may not be perfect.
# helper display function
def imshow(img):
npimg = img.numpy()
plt.imshow(np.transpose(npimg, (1, 2, 0)))
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# obtain one batch of training images
dataiter = iter(celeba_train_loader)
images, _ = dataiter.next() # _ for no labels
# plot the images in the batch, along with the corresponding labels
fig = plt.figure(figsize=(20, 4))
plot_size=20
for idx in np.arange(plot_size):
ax = fig.add_subplot(2, plot_size/2, idx+1, xticks=[], yticks=[])
imshow(images[idx])
<ipython-input-5-13b4d9c7a4be>:17: MatplotlibDeprecationWarning: Passing non-integers as three-element position specification is deprecated since 3.3 and will be removed two minor releases later. ax = fig.add_subplot(2, plot_size/2, idx+1, xticks=[], yticks=[])
You need to do a bit of pre-processing; you know that the output of a tanh activated generator will contain pixel values in a range from -1 to 1, and so, we need to rescale our training images to a range of -1 to 1. (Right now, they are in a range from 0-1.)
# TODO: Complete the scale function
def scale(x, feature_range=(-1, 1)):
''' Scale takes in an image x and returns that image, scaled
with a feature_range of pixel values from -1 to 1.
This function assumes that the input x is already scaled from 0-1.'''
# assume x is scaled to (0, 1)
# scale to feature_range and return scaled x
min, max = feature_range
return x*(max - min) + min
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# check scaled range
# should be close to -1 to 1
img = images[0]
scaled_img = scale(img)
print('Min: ', scaled_img.min())
print('Max: ', scaled_img.max())
Min: tensor(-1.) Max: tensor(0.8196)
A GAN is comprised of two adversarial networks, a discriminator and a generator.
Your first task will be to define the discriminator. This is a convolutional classifier like you've built before, only without any maxpooling layers. To deal with this complex data, it's suggested you use a deep network with normalization. You are also allowed to create any helper functions that may be useful.
import torch.nn as nn
import torch.nn.functional as F
# helper function creating a convolutional layer
def conv(n_inputs, n_outputs, kernel_size, stride=2, padding=1, batch_norm=True):
"""Create convolutional layer with optional batch normalization."""
layers = []
conv_layer = nn.Conv2d(n_inputs, n_outputs, kernel_size, stride, padding, bias=False)
layers.append(conv_layer)
if batch_norm:
# add batch normalization layer
layers.append(nn.BatchNorm2d(n_outputs))
return nn.Sequential(*layers)
class Discriminator(nn.Module):
def __init__(self, conv_dim):
"""
Initialize the Discriminator Module
:param conv_dim: The depth of the first convolutional layer
"""
super(Discriminator, self).__init__()
self.conv_dim = conv_dim
# Output_size = (Input_size - 1) * Strides + Kernel_Size - (2 *padding) + Output Padding
# complete init function
# 32x32x3 input of RGB image
# no batch normalization on the first layer as it is an image
self.conv1 = conv(3, conv_dim, 4, batch_norm=False)
# 16x16xconv_dim out
self.conv2 = conv(conv_dim, conv_dim*2, 4)
# 8x8xconv_dim*2 out
self.conv3 = conv(conv_dim*2, conv_dim*4, 4)
# 4x4xconv_dim*4 out
# final fully connected layer with one output
self.fc = nn.Linear(4*4*conv_dim*4, 1)
self.dropout = nn.Dropout(0.2)
def forward(self, x):
"""
Forward propagation of the neural network
:param x: The input to the neural network
:return: Discriminator logits; the output of the neural network
"""
# define feedforward behavior
# leaky Relu
leaky_slope = 0.2
x = F.leaky_relu(self.conv1(x), leaky_slope)
x = F.leaky_relu(self.conv2(x), leaky_slope)
x = F.leaky_relu(self.conv3(x), leaky_slope)
x = self.dropout(x)
# flatten the output into a vector to be fed into fully connected layer
x = x.view(-1, 4*4*self.conv_dim*4)
# final layer, no Sigmoid as it is in the loss function nn.BCEWithLogitsLoss()
x = F.leaky_relu(self.fc(x))
return x
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_discriminator(Discriminator)
Tests Passed
The generator should upsample an input and generate a new image of the same size as our training data 32x32x3. This should be mostly transpose convolutional layers with normalization applied to the outputs.
z_size32x32x3# helper function creating a transpose convolutional layer
def deconv(n_inputs, n_outputs, kernel_size, stride=2, padding=1, batch_norm=True):
"""Create convolutional layer with optional batch normalization."""
layers = []
transpose_conv_layer = nn.ConvTranspose2d(n_inputs, n_outputs, kernel_size, stride, padding, bias=False)
layers.append(transpose_conv_layer)
if batch_norm:
# add batch normalization layer
layers.append(nn.BatchNorm2d(n_outputs))
return nn.Sequential(*layers)
class Generator(nn.Module):
def __init__(self, z_size, conv_dim):
"""
Initialize the Generator Module
:param z_size: The length of the input latent vector, z
:param conv_dim: The depth of the inputs to the *last* transpose convolutional layer
"""
super(Generator, self).__init__()
# complete init function
self.conv_dim = conv_dim # image dimension as generator output
# input is a latent vector z_size
self.fc = nn.Linear(z_size, 4*4*conv_dim*4)
# transpose convolutional layers
self.trnsp_conv1 = deconv(conv_dim*4, conv_dim*2, 4)
self.trnsp_conv2 = deconv(conv_dim*2, conv_dim, 4)
# no batchnormalization as output is an RGB image
self.trnsp_conv3 = deconv(conv_dim, 3, 4, batch_norm=False)
self.dropout = nn.Dropout(0.2)
def forward(self, x):
"""
Forward propagation of the neural network
:param x: The input to the neural network
:return: A 32x32x3 Tensor image as output
"""
# define feedforward behavior
x = F.relu(self.fc(x))
# reshape into (batch_size, 4, 4)
x = x.view(-1, self.conv_dim*4, 4, 4)
# transpose convolutional layers
x = F.relu(self.trnsp_conv1(x))
x = self.dropout(x)
x = F.relu(self.trnsp_conv2(x))
x = self.dropout(x)
# convert into -1 to 1 image
x = F.tanh(self.trnsp_conv3(x))
return x
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_generator(Generator)
Tests Passed
C:\Users\dmitr\anaconda3\lib\site-packages\torch\nn\functional.py:1698: UserWarning: nn.functional.tanh is deprecated. Use torch.tanh instead.
warnings.warn("nn.functional.tanh is deprecated. Use torch.tanh instead.")
To help your models converge, you should initialize the weights of the convolutional and linear layers in your model. From reading the original DCGAN paper, they say:
All weights were initialized from a zero-centered Normal distribution with standard deviation 0.02.
So, your next task will be to define a weight initialization function that does just this!
You can refer back to the lesson on weight initialization or even consult existing model code, such as that from the networks.py file in CycleGAN Github repository to help you complete this function.
def weights_init_normal(m):
"""
Applies initial weights to certain layers in a model .
The weights are taken from a normal distribution
with mean = 0, std dev = 0.02.
:param m: A module or layer in a network
"""
# classname will be something like:
# `Conv`, `BatchNorm2d`, `Linear`, etc.
mean = 0
std_dev = 0.02
classname = m.__class__.__name__
if classname.find('Conv') != -1 or classname.find('Linear') != -1:
nn.init.normal_(m.weight.data, mean, std_dev)
elif classname.find('BatchNorm2d') != -1:
nn.init.normal_(m.weight.data, 1.0, 0.02)
nn.init.constant_(m.bias.data, 0)
# TODO: Apply initial weights to convolutional and linear layers
Define your models' hyperparameters and instantiate the discriminator and generator from the classes defined above. Make sure you've passed in the correct input arguments.
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
def build_network(d_conv_dim, g_conv_dim, z_size):
# define discriminator and generator
D = Discriminator(d_conv_dim)
G = Generator(z_size=z_size, conv_dim=g_conv_dim)
# initialize model weights
D.apply(weights_init_normal)
G.apply(weights_init_normal)
print(D)
print()
print(G)
return D, G
# Define model hyperparams
d_conv_dim = 128
g_conv_dim = 128
z_size = 128
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
D, G = build_network(d_conv_dim, g_conv_dim, z_size)
Discriminator(
(conv1): Sequential(
(0): Conv2d(3, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
)
(conv2): Sequential(
(0): Conv2d(128, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(conv3): Sequential(
(0): Conv2d(256, 512, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(fc): Linear(in_features=8192, out_features=1, bias=True)
(dropout): Dropout(p=0.2, inplace=False)
)
Generator(
(fc): Linear(in_features=128, out_features=8192, bias=True)
(trnsp_conv1): Sequential(
(0): ConvTranspose2d(512, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(trnsp_conv2): Sequential(
(0): ConvTranspose2d(256, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(trnsp_conv3): Sequential(
(0): ConvTranspose2d(128, 3, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
)
(dropout): Dropout(p=0.2, inplace=False)
)
Check if you can train on GPU. Here, we'll set this as a boolean variable train_on_gpu. Later, you'll be responsible for making sure that
- Models,
- Model inputs, and
- Loss function arguments
Are moved to GPU, where appropriate.
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import torch
# Check for a GPU
train_on_gpu = torch.cuda.is_available()
if not train_on_gpu:
print('No GPU found. Please use a GPU to train your neural network.')
else:
print('Training on GPU!')
Training on GPU!
Now we need to calculate the losses for both types of adversarial networks.
- For the discriminator, the total loss is the sum of the losses for real and fake images,
d_loss = d_real_loss + d_fake_loss.- Remember that we want the discriminator to output 1 for real images and 0 for fake images, so we need to set up the losses to reflect that.
The generator loss will look similar only with flipped labels. The generator's goal is to get the discriminator to think its generated images are real.
You may choose to use either cross entropy or a least squares error loss to complete the following real_loss and fake_loss functions.
def real_loss(D_out):
'''Calculates how close discriminator outputs are to being real.
param, D_out: discriminator logits
return: real loss'''
batch_size = D_out.size(0)
labels = torch.ones(batch_size)*0.9 # use smoothing
if train_on_gpu:
labels = labels.cuda()
criterion = nn.BCEWithLogitsLoss()
loss = criterion(D_out.squeeze(), labels)
return loss
def fake_loss(D_out):
'''Calculates how close discriminator outputs are to being fake.
param, D_out: discriminator logits
return: fake loss'''
batch_size = D_out.size(0)
labels = torch.zeros(batch_size)
if train_on_gpu:
labels = labels.cuda()
criterion = nn.BCEWithLogitsLoss()
loss = criterion(D_out.squeeze(), labels)
return loss
import torch.optim as optim
learning_rate = 0.0002
beta1 = 0.5
beta2 = 0.999
# Create optimizers for the discriminator D and generator G
d_optimizer = optim.Adam(D.parameters(), learning_rate, [beta1, beta2])
g_optimizer = optim.Adam(G.parameters(), learning_rate, [beta1, beta2])
Training will involve alternating between training the discriminator and the generator. You'll use your functions real_loss and fake_loss to help you calculate the discriminator losses.
You've been given some code to print out some loss statistics and save some generated "fake" samples.
Keep in mind that, if you've moved your models to GPU, you'll also have to move any model inputs to GPU.
def train(D, G, n_epochs, print_every=50):
'''Trains adversarial networks for some number of epochs
param, D: the discriminator network
param, G: the generator network
param, n_epochs: number of epochs to train for
param, print_every: when to print and record the models' losses
return: D and G losses'''
# move models to GPU
if train_on_gpu:
D.cuda()
G.cuda()
# keep track of loss and generated, "fake" samples
samples = []
losses = []
# Get some fixed data for sampling. These are images that are held
# constant throughout training, and allow us to inspect the model's performance
sample_size=16
fixed_z = np.random.uniform(-1, 1, size=(sample_size, z_size))
fixed_z = torch.from_numpy(fixed_z).float()
# move z to GPU if available
if train_on_gpu:
fixed_z = fixed_z.cuda()
# epoch training loop
for epoch in range(n_epochs):
# batch training loop
for batch_i, (real_images, _) in enumerate(celeba_train_loader):
batch_size = real_images.size(0)
real_images = scale(real_images)
# ===============================================
# YOUR CODE HERE: TRAIN THE NETWORKS
# ===============================================
# 1. Train the discriminator on real and fake images
d_optimizer.zero_grad()
# do real images
if train_on_gpu:
real_images = real_images.cuda()
D_real = D(real_images)
d_real_loss = real_loss(D_real)
# do fake images
z = np.random.uniform(-1, 1, size=(batch_size, z_size))
z = torch.from_numpy(z).float()
if train_on_gpu:
z = z.cuda()
fake_images = G(z)
D_fake = D(fake_images)
d_fake_loss = fake_loss(D_fake)
d_loss = d_real_loss + d_fake_loss
d_loss.backward()
d_optimizer.step()
# 2. Train the generator with an adversarial loss
g_optimizer.zero_grad()
z = np.random.uniform(-1, 1, size=(batch_size, z_size))
z = torch.from_numpy(z).float()
if train_on_gpu:
z = z.cuda()
fake_images = G(z)
D_fake = D(fake_images)
g_loss = real_loss(D_fake)
g_loss.backward()
g_optimizer.step()
# ===============================================
# END OF YOUR CODE
# ===============================================
# Print some loss stats
if batch_i % print_every == 0:
# append discriminator loss and generator loss
losses.append((d_loss.item(), g_loss.item()))
# print discriminator and generator loss
print('Epoch [{:5d}/{:5d}] | d_loss: {:6.4f} | g_loss: {:6.4f}'.format(
epoch+1, n_epochs, d_loss.item(), g_loss.item()))
## AFTER EACH EPOCH##
# this code assumes your generator is named G, feel free to change the name
# generate and save sample, fake images
G.eval() # for generating samples
samples_z = G(fixed_z)
samples.append(samples_z)
G.train() # back to training mode
# Save training generator samples
with open('train_samples.pkl', 'wb') as f:
pkl.dump(samples, f)
# finally return losses
return losses
Set your number of training epochs and train your GAN!
from time import time
# set number of epochs
n_epochs = 10
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# call training function
time_start = time()
losses = train(D, G, n_epochs=n_epochs)
print(f"Training time: {time() - time_start:.1f} seconds")
C:\Users\dmitr\anaconda3\lib\site-packages\torch\nn\functional.py:1698: UserWarning: nn.functional.tanh is deprecated. Use torch.tanh instead.
warnings.warn("nn.functional.tanh is deprecated. Use torch.tanh instead.")
Epoch [ 1/ 10] | d_loss: 1.3318 | g_loss: 0.8145 Epoch [ 1/ 10] | d_loss: 1.3355 | g_loss: 0.8402 Epoch [ 1/ 10] | d_loss: 1.3243 | g_loss: 0.8189 Epoch [ 1/ 10] | d_loss: 1.3474 | g_loss: 0.8518 Epoch [ 1/ 10] | d_loss: 1.3383 | g_loss: 0.8184 Epoch [ 1/ 10] | d_loss: 1.3418 | g_loss: 0.9352 Epoch [ 1/ 10] | d_loss: 1.3552 | g_loss: 0.8262 Epoch [ 1/ 10] | d_loss: 1.3051 | g_loss: 0.8524 Epoch [ 1/ 10] | d_loss: 1.5071 | g_loss: 0.8586 Epoch [ 1/ 10] | d_loss: 1.3400 | g_loss: 0.8140 Epoch [ 1/ 10] | d_loss: 1.3751 | g_loss: 0.8755 Epoch [ 1/ 10] | d_loss: 1.3631 | g_loss: 0.8178 Epoch [ 1/ 10] | d_loss: 1.3514 | g_loss: 0.8284 Epoch [ 1/ 10] | d_loss: 1.3501 | g_loss: 0.8461 Epoch [ 1/ 10] | d_loss: 1.3236 | g_loss: 0.8430 Epoch [ 1/ 10] | d_loss: 1.3698 | g_loss: 0.7907 Epoch [ 1/ 10] | d_loss: 1.3688 | g_loss: 0.8074 Epoch [ 1/ 10] | d_loss: 1.3588 | g_loss: 0.8068 Epoch [ 1/ 10] | d_loss: 1.3446 | g_loss: 0.8118 Epoch [ 1/ 10] | d_loss: 1.3420 | g_loss: 0.8545 Epoch [ 1/ 10] | d_loss: 1.3159 | g_loss: 0.8467 Epoch [ 1/ 10] | d_loss: 1.3104 | g_loss: 0.8145 Epoch [ 1/ 10] | d_loss: 1.3534 | g_loss: 0.8191 Epoch [ 1/ 10] | d_loss: 1.3555 | g_loss: 0.8160 Epoch [ 1/ 10] | d_loss: 1.3189 | g_loss: 0.8483 Epoch [ 1/ 10] | d_loss: 1.3759 | g_loss: 0.8377 Epoch [ 1/ 10] | d_loss: 1.3622 | g_loss: 0.8029 Epoch [ 1/ 10] | d_loss: 1.3584 | g_loss: 0.8176 Epoch [ 1/ 10] | d_loss: 1.3371 | g_loss: 0.8567 Epoch [ 1/ 10] | d_loss: 1.3821 | g_loss: 0.8028 Epoch [ 1/ 10] | d_loss: 1.4066 | g_loss: 0.8209 Epoch [ 1/ 10] | d_loss: 1.3779 | g_loss: 0.7919 Epoch [ 1/ 10] | d_loss: 1.3702 | g_loss: 0.8294 Epoch [ 1/ 10] | d_loss: 1.3328 | g_loss: 0.8400 Epoch [ 1/ 10] | d_loss: 1.3634 | g_loss: 0.8130 Epoch [ 1/ 10] | d_loss: 1.3731 | g_loss: 0.8309 Epoch [ 1/ 10] | d_loss: 1.3583 | g_loss: 0.8183 Epoch [ 1/ 10] | d_loss: 1.3642 | g_loss: 0.8680 Epoch [ 1/ 10] | d_loss: 1.3634 | g_loss: 0.8053 Epoch [ 1/ 10] | d_loss: 1.3567 | g_loss: 0.8037 Epoch [ 1/ 10] | d_loss: 1.3876 | g_loss: 0.8544 Epoch [ 1/ 10] | d_loss: 1.3605 | g_loss: 0.8495 Epoch [ 1/ 10] | d_loss: 1.3050 | g_loss: 0.8680 Epoch [ 1/ 10] | d_loss: 1.3270 | g_loss: 0.8216 Epoch [ 1/ 10] | d_loss: 1.3342 | g_loss: 0.8390 Epoch [ 1/ 10] | d_loss: 1.3293 | g_loss: 0.8508 Epoch [ 1/ 10] | d_loss: 1.3742 | g_loss: 0.7284 Epoch [ 1/ 10] | d_loss: 1.3602 | g_loss: 0.9399 Epoch [ 1/ 10] | d_loss: 1.3585 | g_loss: 0.8384 Epoch [ 1/ 10] | d_loss: 1.3452 | g_loss: 0.8486 Epoch [ 1/ 10] | d_loss: 1.3204 | g_loss: 0.8437 Epoch [ 1/ 10] | d_loss: 1.3397 | g_loss: 0.8306 Epoch [ 1/ 10] | d_loss: 1.3531 | g_loss: 0.8317 Epoch [ 1/ 10] | d_loss: 1.3403 | g_loss: 0.8117 Epoch [ 1/ 10] | d_loss: 1.3853 | g_loss: 0.8291 Epoch [ 1/ 10] | d_loss: 1.3850 | g_loss: 0.8391 Epoch [ 1/ 10] | d_loss: 1.3187 | g_loss: 0.8745 Epoch [ 2/ 10] | d_loss: 1.3439 | g_loss: 0.8310 Epoch [ 2/ 10] | d_loss: 1.3834 | g_loss: 0.7990 Epoch [ 2/ 10] | d_loss: 1.3636 | g_loss: 0.8153 Epoch [ 2/ 10] | d_loss: 1.3295 | g_loss: 0.8529 Epoch [ 2/ 10] | d_loss: 1.2666 | g_loss: 0.8280 Epoch [ 2/ 10] | d_loss: 1.3235 | g_loss: 0.8445 Epoch [ 2/ 10] | d_loss: 1.3271 | g_loss: 0.8516 Epoch [ 2/ 10] | d_loss: 1.2959 | g_loss: 0.8185 Epoch [ 2/ 10] | d_loss: 1.3026 | g_loss: 0.7852 Epoch [ 2/ 10] | d_loss: 1.3432 | g_loss: 0.8407 Epoch [ 2/ 10] | d_loss: 1.3620 | g_loss: 0.8136 Epoch [ 2/ 10] | d_loss: 1.3622 | g_loss: 0.8616 Epoch [ 2/ 10] | d_loss: 1.3596 | g_loss: 0.8529 Epoch [ 2/ 10] | d_loss: 1.3778 | g_loss: 0.8284 Epoch [ 2/ 10] | d_loss: 1.3854 | g_loss: 0.7820 Epoch [ 2/ 10] | d_loss: 1.3684 | g_loss: 0.7866 Epoch [ 2/ 10] | d_loss: 1.3736 | g_loss: 0.7902 Epoch [ 2/ 10] | d_loss: 1.3639 | g_loss: 0.8080 Epoch [ 2/ 10] | d_loss: 1.3761 | g_loss: 0.7995 Epoch [ 2/ 10] | d_loss: 1.3731 | g_loss: 0.8176 Epoch [ 2/ 10] | d_loss: 1.3580 | g_loss: 0.8243 Epoch [ 2/ 10] | d_loss: 1.3440 | g_loss: 0.8757 Epoch [ 2/ 10] | d_loss: 1.3026 | g_loss: 0.8382 Epoch [ 2/ 10] | d_loss: 1.3804 | g_loss: 0.8523 Epoch [ 2/ 10] | d_loss: 1.3131 | g_loss: 0.8563 Epoch [ 2/ 10] | d_loss: 1.3970 | g_loss: 0.8095 Epoch [ 2/ 10] | d_loss: 1.3602 | g_loss: 0.8091 Epoch [ 2/ 10] | d_loss: 1.3646 | g_loss: 0.7958 Epoch [ 2/ 10] | d_loss: 1.3544 | g_loss: 0.8627 Epoch [ 2/ 10] | d_loss: 1.3295 | g_loss: 0.8566 Epoch [ 2/ 10] | d_loss: 1.3770 | g_loss: 0.8404 Epoch [ 2/ 10] | d_loss: 1.3641 | g_loss: 0.8442 Epoch [ 2/ 10] | d_loss: 1.3745 | g_loss: 0.8324 Epoch [ 2/ 10] | d_loss: 1.3549 | g_loss: 0.8390 Epoch [ 2/ 10] | d_loss: 1.3432 | g_loss: 0.8182 Epoch [ 2/ 10] | d_loss: 1.3627 | g_loss: 0.8027 Epoch [ 2/ 10] | d_loss: 1.3353 | g_loss: 0.8516 Epoch [ 2/ 10] | d_loss: 1.3228 | g_loss: 0.8596 Epoch [ 2/ 10] | d_loss: 1.3538 | g_loss: 0.9008 Epoch [ 2/ 10] | d_loss: 1.3910 | g_loss: 0.8004 Epoch [ 2/ 10] | d_loss: 1.4128 | g_loss: 0.7817 Epoch [ 2/ 10] | d_loss: 1.3770 | g_loss: 0.7968 Epoch [ 2/ 10] | d_loss: 1.3706 | g_loss: 0.8225 Epoch [ 2/ 10] | d_loss: 1.3944 | g_loss: 0.8187 Epoch [ 2/ 10] | d_loss: 1.3907 | g_loss: 0.8072 Epoch [ 2/ 10] | d_loss: 1.3489 | g_loss: 0.7978 Epoch [ 2/ 10] | d_loss: 1.3720 | g_loss: 0.8142 Epoch [ 2/ 10] | d_loss: 1.3783 | g_loss: 0.8102 Epoch [ 2/ 10] | d_loss: 1.3452 | g_loss: 0.8160 Epoch [ 2/ 10] | d_loss: 1.3547 | g_loss: 0.8131 Epoch [ 2/ 10] | d_loss: 1.3549 | g_loss: 0.8540 Epoch [ 2/ 10] | d_loss: 1.4011 | g_loss: 0.8675 Epoch [ 2/ 10] | d_loss: 1.3222 | g_loss: 0.8515 Epoch [ 2/ 10] | d_loss: 1.3459 | g_loss: 0.8420 Epoch [ 2/ 10] | d_loss: 1.3401 | g_loss: 0.8297 Epoch [ 2/ 10] | d_loss: 1.3969 | g_loss: 0.8568 Epoch [ 2/ 10] | d_loss: 1.3483 | g_loss: 0.7919 Epoch [ 3/ 10] | d_loss: 1.3768 | g_loss: 0.7982 Epoch [ 3/ 10] | d_loss: 1.3374 | g_loss: 0.8628 Epoch [ 3/ 10] | d_loss: 1.3333 | g_loss: 0.8641 Epoch [ 3/ 10] | d_loss: 1.3261 | g_loss: 0.8072 Epoch [ 3/ 10] | d_loss: 1.3660 | g_loss: 0.8610 Epoch [ 3/ 10] | d_loss: 1.3306 | g_loss: 0.8528 Epoch [ 3/ 10] | d_loss: 1.3741 | g_loss: 0.8580 Epoch [ 3/ 10] | d_loss: 1.3917 | g_loss: 0.8045 Epoch [ 3/ 10] | d_loss: 1.3868 | g_loss: 0.7949 Epoch [ 3/ 10] | d_loss: 1.3974 | g_loss: 0.7990 Epoch [ 3/ 10] | d_loss: 1.3765 | g_loss: 0.7923 Epoch [ 3/ 10] | d_loss: 1.3724 | g_loss: 0.7811 Epoch [ 3/ 10] | d_loss: 1.3772 | g_loss: 0.7883 Epoch [ 3/ 10] | d_loss: 1.3698 | g_loss: 0.8118 Epoch [ 3/ 10] | d_loss: 1.3733 | g_loss: 0.8211 Epoch [ 3/ 10] | d_loss: 1.3606 | g_loss: 0.8100 Epoch [ 3/ 10] | d_loss: 1.3657 | g_loss: 0.8140 Epoch [ 3/ 10] | d_loss: 1.3454 | g_loss: 0.8176 Epoch [ 3/ 10] | d_loss: 1.3518 | g_loss: 0.8134 Epoch [ 3/ 10] | d_loss: 1.3523 | g_loss: 0.8191 Epoch [ 3/ 10] | d_loss: 1.3860 | g_loss: 0.9303 Epoch [ 3/ 10] | d_loss: 1.3626 | g_loss: 0.8511 Epoch [ 3/ 10] | d_loss: 1.3521 | g_loss: 0.8565 Epoch [ 3/ 10] | d_loss: 1.3665 | g_loss: 0.8128 Epoch [ 3/ 10] | d_loss: 1.3007 | g_loss: 0.8294 Epoch [ 3/ 10] | d_loss: 1.3590 | g_loss: 0.7876 Epoch [ 3/ 10] | d_loss: 1.3787 | g_loss: 0.7997 Epoch [ 3/ 10] | d_loss: 1.3598 | g_loss: 0.8431 Epoch [ 3/ 10] | d_loss: 1.3311 | g_loss: 0.8690 Epoch [ 3/ 10] | d_loss: 1.3436 | g_loss: 0.8412 Epoch [ 3/ 10] | d_loss: 1.3512 | g_loss: 0.8574 Epoch [ 3/ 10] | d_loss: 1.3437 | g_loss: 0.8344 Epoch [ 3/ 10] | d_loss: 1.3886 | g_loss: 0.8251 Epoch [ 3/ 10] | d_loss: 1.4862 | g_loss: 0.8410 Epoch [ 3/ 10] | d_loss: 1.3832 | g_loss: 0.8198 Epoch [ 3/ 10] | d_loss: 1.3618 | g_loss: 0.8751 Epoch [ 3/ 10] | d_loss: 1.3473 | g_loss: 0.8370 Epoch [ 3/ 10] | d_loss: 1.3537 | g_loss: 0.8271 Epoch [ 3/ 10] | d_loss: 1.3121 | g_loss: 0.8459 Epoch [ 3/ 10] | d_loss: 1.3972 | g_loss: 0.8590 Epoch [ 3/ 10] | d_loss: 1.3650 | g_loss: 0.8100 Epoch [ 3/ 10] | d_loss: 1.3787 | g_loss: 0.8736 Epoch [ 3/ 10] | d_loss: 1.3771 | g_loss: 0.7962 Epoch [ 3/ 10] | d_loss: 1.3772 | g_loss: 0.8099 Epoch [ 3/ 10] | d_loss: 1.3758 | g_loss: 0.8061 Epoch [ 3/ 10] | d_loss: 1.3793 | g_loss: 0.7890 Epoch [ 3/ 10] | d_loss: 1.3604 | g_loss: 0.8351 Epoch [ 3/ 10] | d_loss: 1.3655 | g_loss: 0.7896 Epoch [ 3/ 10] | d_loss: 1.3845 | g_loss: 0.8602 Epoch [ 3/ 10] | d_loss: 1.3729 | g_loss: 0.8090 Epoch [ 3/ 10] | d_loss: 1.3832 | g_loss: 0.8634 Epoch [ 3/ 10] | d_loss: 1.3425 | g_loss: 0.8220 Epoch [ 3/ 10] | d_loss: 1.3665 | g_loss: 0.8376 Epoch [ 3/ 10] | d_loss: 1.3184 | g_loss: 0.8173 Epoch [ 3/ 10] | d_loss: 1.3277 | g_loss: 0.8412 Epoch [ 3/ 10] | d_loss: 1.3817 | g_loss: 0.8928 Epoch [ 3/ 10] | d_loss: 1.4081 | g_loss: 0.8699 Epoch [ 4/ 10] | d_loss: 1.3366 | g_loss: 0.8630 Epoch [ 4/ 10] | d_loss: 1.3471 | g_loss: 0.8575 Epoch [ 4/ 10] | d_loss: 1.3551 | g_loss: 0.8190 Epoch [ 4/ 10] | d_loss: 1.3201 | g_loss: 0.8088 Epoch [ 4/ 10] | d_loss: 1.3909 | g_loss: 0.7886 Epoch [ 4/ 10] | d_loss: 1.4055 | g_loss: 0.8488 Epoch [ 4/ 10] | d_loss: 1.3741 | g_loss: 0.8114 Epoch [ 4/ 10] | d_loss: 1.3762 | g_loss: 0.7995 Epoch [ 4/ 10] | d_loss: 1.3555 | g_loss: 0.8108 Epoch [ 4/ 10] | d_loss: 1.3608 | g_loss: 0.8185 Epoch [ 4/ 10] | d_loss: 1.3731 | g_loss: 0.8353 Epoch [ 4/ 10] | d_loss: 1.3204 | g_loss: 0.8722 Epoch [ 4/ 10] | d_loss: 1.3322 | g_loss: 0.8391 Epoch [ 4/ 10] | d_loss: 1.3272 | g_loss: 0.8482 Epoch [ 4/ 10] | d_loss: 1.2970 | g_loss: 0.8512 Epoch [ 4/ 10] | d_loss: 1.3377 | g_loss: 0.8543 Epoch [ 4/ 10] | d_loss: 1.3731 | g_loss: 0.8637 Epoch [ 4/ 10] | d_loss: 1.3528 | g_loss: 0.8181 Epoch [ 4/ 10] | d_loss: 1.3307 | g_loss: 0.8610 Epoch [ 4/ 10] | d_loss: 1.3533 | g_loss: 0.8256 Epoch [ 4/ 10] | d_loss: 1.3807 | g_loss: 0.8327 Epoch [ 4/ 10] | d_loss: 1.3126 | g_loss: 0.8450 Epoch [ 4/ 10] | d_loss: 1.3720 | g_loss: 0.8328 Epoch [ 4/ 10] | d_loss: 1.3748 | g_loss: 0.8208 Epoch [ 4/ 10] | d_loss: 1.3987 | g_loss: 0.8507 Epoch [ 4/ 10] | d_loss: 1.3890 | g_loss: 0.8174 Epoch [ 4/ 10] | d_loss: 1.3840 | g_loss: 0.7941 Epoch [ 4/ 10] | d_loss: 1.3839 | g_loss: 0.7933 Epoch [ 4/ 10] | d_loss: 1.3884 | g_loss: 0.8125 Epoch [ 4/ 10] | d_loss: 1.3779 | g_loss: 0.8083 Epoch [ 4/ 10] | d_loss: 1.3902 | g_loss: 0.8088 Epoch [ 4/ 10] | d_loss: 1.3806 | g_loss: 0.8088 Epoch [ 4/ 10] | d_loss: 1.3947 | g_loss: 0.7885 Epoch [ 4/ 10] | d_loss: 1.3904 | g_loss: 0.7900 Epoch [ 4/ 10] | d_loss: 1.3754 | g_loss: 0.7872 Epoch [ 4/ 10] | d_loss: 1.3773 | g_loss: 0.7924 Epoch [ 4/ 10] | d_loss: 1.3760 | g_loss: 0.7871 Epoch [ 4/ 10] | d_loss: 1.3822 | g_loss: 0.8072 Epoch [ 4/ 10] | d_loss: 1.3615 | g_loss: 0.7966 Epoch [ 4/ 10] | d_loss: 1.3650 | g_loss: 0.8124 Epoch [ 4/ 10] | d_loss: 1.3645 | g_loss: 0.8167 Epoch [ 4/ 10] | d_loss: 1.3304 | g_loss: 0.8214 Epoch [ 4/ 10] | d_loss: 1.3717 | g_loss: 0.8272 Epoch [ 4/ 10] | d_loss: 1.3950 | g_loss: 0.8232 Epoch [ 4/ 10] | d_loss: 1.3544 | g_loss: 0.8155 Epoch [ 4/ 10] | d_loss: 1.3829 | g_loss: 0.8308 Epoch [ 4/ 10] | d_loss: 1.3830 | g_loss: 0.8051 Epoch [ 4/ 10] | d_loss: 1.3918 | g_loss: 0.8168 Epoch [ 4/ 10] | d_loss: 1.3799 | g_loss: 0.8028 Epoch [ 4/ 10] | d_loss: 1.3885 | g_loss: 0.8036 Epoch [ 4/ 10] | d_loss: 1.3814 | g_loss: 0.8191 Epoch [ 4/ 10] | d_loss: 1.3921 | g_loss: 0.8418 Epoch [ 4/ 10] | d_loss: 1.3688 | g_loss: 0.8075 Epoch [ 4/ 10] | d_loss: 1.3658 | g_loss: 0.8199 Epoch [ 4/ 10] | d_loss: 1.3589 | g_loss: 0.8396 Epoch [ 4/ 10] | d_loss: 1.3395 | g_loss: 0.8527 Epoch [ 4/ 10] | d_loss: 1.3495 | g_loss: 0.8621 Epoch [ 5/ 10] | d_loss: 1.3194 | g_loss: 0.8407 Epoch [ 5/ 10] | d_loss: 1.3323 | g_loss: 0.8349 Epoch [ 5/ 10] | d_loss: 1.3059 | g_loss: 0.8412 Epoch [ 5/ 10] | d_loss: 1.3811 | g_loss: 0.8223 Epoch [ 5/ 10] | d_loss: 1.3561 | g_loss: 0.7951 Epoch [ 5/ 10] | d_loss: 1.3948 | g_loss: 0.8109 Epoch [ 5/ 10] | d_loss: 1.3604 | g_loss: 0.8273 Epoch [ 5/ 10] | d_loss: 1.3552 | g_loss: 0.8470 Epoch [ 5/ 10] | d_loss: 1.3545 | g_loss: 0.8211 Epoch [ 5/ 10] | d_loss: 1.3583 | g_loss: 0.8491 Epoch [ 5/ 10] | d_loss: 1.3464 | g_loss: 0.8500 Epoch [ 5/ 10] | d_loss: 1.3916 | g_loss: 0.8102 Epoch [ 5/ 10] | d_loss: 1.3549 | g_loss: 0.8290 Epoch [ 5/ 10] | d_loss: 1.3281 | g_loss: 0.8459 Epoch [ 5/ 10] | d_loss: 1.3585 | g_loss: 0.8175 Epoch [ 5/ 10] | d_loss: 1.3616 | g_loss: 0.8396 Epoch [ 5/ 10] | d_loss: 1.3609 | g_loss: 0.8297 Epoch [ 5/ 10] | d_loss: 1.2996 | g_loss: 0.8154 Epoch [ 5/ 10] | d_loss: 1.3927 | g_loss: 0.8214 Epoch [ 5/ 10] | d_loss: 1.3890 | g_loss: 0.7873 Epoch [ 5/ 10] | d_loss: 1.3746 | g_loss: 0.7893 Epoch [ 5/ 10] | d_loss: 1.3792 | g_loss: 0.8011 Epoch [ 5/ 10] | d_loss: 1.3755 | g_loss: 0.7784 Epoch [ 5/ 10] | d_loss: 1.3737 | g_loss: 0.7964 Epoch [ 5/ 10] | d_loss: 1.3776 | g_loss: 0.7865 Epoch [ 5/ 10] | d_loss: 1.3778 | g_loss: 0.7846 Epoch [ 5/ 10] | d_loss: 1.3793 | g_loss: 0.7804 Epoch [ 5/ 10] | d_loss: 1.3802 | g_loss: 0.7873 Epoch [ 5/ 10] | d_loss: 1.3739 | g_loss: 0.7906 Epoch [ 5/ 10] | d_loss: 1.3770 | g_loss: 0.8053 Epoch [ 5/ 10] | d_loss: 1.3828 | g_loss: 0.7917 Epoch [ 5/ 10] | d_loss: 1.3597 | g_loss: 0.7869 Epoch [ 5/ 10] | d_loss: 1.3750 | g_loss: 0.7949 Epoch [ 5/ 10] | d_loss: 1.3698 | g_loss: 0.7964 Epoch [ 5/ 10] | d_loss: 1.3758 | g_loss: 0.7889 Epoch [ 5/ 10] | d_loss: 1.3671 | g_loss: 0.7870 Epoch [ 5/ 10] | d_loss: 1.3729 | g_loss: 0.8010 Epoch [ 5/ 10] | d_loss: 1.3724 | g_loss: 0.8061 Epoch [ 5/ 10] | d_loss: 1.3744 | g_loss: 0.8232 Epoch [ 5/ 10] | d_loss: 1.3631 | g_loss: 0.7888 Epoch [ 5/ 10] | d_loss: 1.3474 | g_loss: 0.8108 Epoch [ 5/ 10] | d_loss: 1.3588 | g_loss: 0.8539 Epoch [ 5/ 10] | d_loss: 1.3516 | g_loss: 0.8759 Epoch [ 5/ 10] | d_loss: 1.3533 | g_loss: 0.8655 Epoch [ 5/ 10] | d_loss: 1.3830 | g_loss: 0.8620 Epoch [ 5/ 10] | d_loss: 1.3751 | g_loss: 0.8120 Epoch [ 5/ 10] | d_loss: 1.3562 | g_loss: 0.8080 Epoch [ 5/ 10] | d_loss: 1.3727 | g_loss: 0.7941 Epoch [ 5/ 10] | d_loss: 1.3883 | g_loss: 0.7888 Epoch [ 5/ 10] | d_loss: 1.3827 | g_loss: 0.8226 Epoch [ 5/ 10] | d_loss: 1.3804 | g_loss: 0.8251 Epoch [ 5/ 10] | d_loss: 1.3453 | g_loss: 0.8216 Epoch [ 5/ 10] | d_loss: 1.3886 | g_loss: 0.8650 Epoch [ 5/ 10] | d_loss: 1.3514 | g_loss: 0.8169 Epoch [ 5/ 10] | d_loss: 1.3309 | g_loss: 0.8825 Epoch [ 5/ 10] | d_loss: 1.3471 | g_loss: 0.8405 Epoch [ 5/ 10] | d_loss: 1.3447 | g_loss: 0.8280 Epoch [ 6/ 10] | d_loss: 1.2691 | g_loss: 0.7382 Epoch [ 6/ 10] | d_loss: 1.4140 | g_loss: 0.8654 Epoch [ 6/ 10] | d_loss: 1.4036 | g_loss: 0.8256 Epoch [ 6/ 10] | d_loss: 1.3809 | g_loss: 0.8100 Epoch [ 6/ 10] | d_loss: 1.3669 | g_loss: 0.8013 Epoch [ 6/ 10] | d_loss: 1.3756 | g_loss: 0.7950 Epoch [ 6/ 10] | d_loss: 1.3775 | g_loss: 0.7987 Epoch [ 6/ 10] | d_loss: 1.3798 | g_loss: 0.8071 Epoch [ 6/ 10] | d_loss: 1.3771 | g_loss: 0.8027 Epoch [ 6/ 10] | d_loss: 1.3719 | g_loss: 0.7953 Epoch [ 6/ 10] | d_loss: 1.3537 | g_loss: 0.8071 Epoch [ 6/ 10] | d_loss: 1.3635 | g_loss: 0.8323 Epoch [ 6/ 10] | d_loss: 1.3691 | g_loss: 0.8207 Epoch [ 6/ 10] | d_loss: 1.3598 | g_loss: 0.8324 Epoch [ 6/ 10] | d_loss: 1.3515 | g_loss: 0.8488 Epoch [ 6/ 10] | d_loss: 1.3553 | g_loss: 0.8527 Epoch [ 6/ 10] | d_loss: 1.3269 | g_loss: 0.8495 Epoch [ 6/ 10] | d_loss: 1.3397 | g_loss: 0.8656 Epoch [ 6/ 10] | d_loss: 1.3418 | g_loss: 0.8106 Epoch [ 6/ 10] | d_loss: 1.3126 | g_loss: 0.8327 Epoch [ 6/ 10] | d_loss: 1.3356 | g_loss: 0.8473 Epoch [ 6/ 10] | d_loss: 1.4031 | g_loss: 0.8158 Epoch [ 6/ 10] | d_loss: 1.3748 | g_loss: 0.7743 Epoch [ 6/ 10] | d_loss: 1.3791 | g_loss: 0.7830 Epoch [ 6/ 10] | d_loss: 1.3671 | g_loss: 0.8005 Epoch [ 6/ 10] | d_loss: 1.3667 | g_loss: 0.8034 Epoch [ 6/ 10] | d_loss: 1.3760 | g_loss: 0.7972 Epoch [ 6/ 10] | d_loss: 1.3678 | g_loss: 0.8002 Epoch [ 6/ 10] | d_loss: 1.3785 | g_loss: 0.7961 Epoch [ 6/ 10] | d_loss: 1.3859 | g_loss: 0.7923 Epoch [ 6/ 10] | d_loss: 1.3539 | g_loss: 0.7981 Epoch [ 6/ 10] | d_loss: 1.3645 | g_loss: 0.8237 Epoch [ 6/ 10] | d_loss: 1.3648 | g_loss: 0.8034 Epoch [ 6/ 10] | d_loss: 1.3432 | g_loss: 0.8043 Epoch [ 6/ 10] | d_loss: 1.3494 | g_loss: 0.8373 Epoch [ 6/ 10] | d_loss: 1.3407 | g_loss: 0.8120 Epoch [ 6/ 10] | d_loss: 1.3402 | g_loss: 0.8541 Epoch [ 6/ 10] | d_loss: 1.3611 | g_loss: 0.8575 Epoch [ 6/ 10] | d_loss: 1.3500 | g_loss: 0.8493 Epoch [ 6/ 10] | d_loss: 1.2896 | g_loss: 0.8085 Epoch [ 6/ 10] | d_loss: 1.3559 | g_loss: 0.8344 Epoch [ 6/ 10] | d_loss: 1.3523 | g_loss: 0.8480 Epoch [ 6/ 10] | d_loss: 1.3219 | g_loss: 0.8433 Epoch [ 6/ 10] | d_loss: 1.3798 | g_loss: 0.9158 Epoch [ 6/ 10] | d_loss: 1.3802 | g_loss: 0.7888 Epoch [ 6/ 10] | d_loss: 1.3887 | g_loss: 0.7867 Epoch [ 6/ 10] | d_loss: 1.3769 | g_loss: 0.8052 Epoch [ 6/ 10] | d_loss: 1.3709 | g_loss: 0.8470 Epoch [ 6/ 10] | d_loss: 1.3958 | g_loss: 0.8003 Epoch [ 6/ 10] | d_loss: 1.3647 | g_loss: 0.8080 Epoch [ 6/ 10] | d_loss: 1.3638 | g_loss: 0.8133 Epoch [ 6/ 10] | d_loss: 1.3582 | g_loss: 0.8020 Epoch [ 6/ 10] | d_loss: 1.3674 | g_loss: 0.8243 Epoch [ 6/ 10] | d_loss: 1.3714 | g_loss: 0.8319 Epoch [ 6/ 10] | d_loss: 1.3337 | g_loss: 0.8952 Epoch [ 6/ 10] | d_loss: 1.2507 | g_loss: 0.7750 Epoch [ 6/ 10] | d_loss: 1.3697 | g_loss: 0.8128 Epoch [ 7/ 10] | d_loss: 1.3726 | g_loss: 0.8111 Epoch [ 7/ 10] | d_loss: 1.3617 | g_loss: 0.7910 Epoch [ 7/ 10] | d_loss: 1.3578 | g_loss: 0.8248 Epoch [ 7/ 10] | d_loss: 1.3578 | g_loss: 0.9254 Epoch [ 7/ 10] | d_loss: 1.3581 | g_loss: 0.8561 Epoch [ 7/ 10] | d_loss: 1.3405 | g_loss: 0.8296 Epoch [ 7/ 10] | d_loss: 1.3085 | g_loss: 0.8463 Epoch [ 7/ 10] | d_loss: 1.3701 | g_loss: 0.8165 Epoch [ 7/ 10] | d_loss: 1.3789 | g_loss: 0.8301 Epoch [ 7/ 10] | d_loss: 1.3675 | g_loss: 0.8146 Epoch [ 7/ 10] | d_loss: 1.3474 | g_loss: 0.8718 Epoch [ 7/ 10] | d_loss: 1.3583 | g_loss: 0.8444 Epoch [ 7/ 10] | d_loss: 1.4078 | g_loss: 0.8288 Epoch [ 7/ 10] | d_loss: 1.3870 | g_loss: 0.7973 Epoch [ 7/ 10] | d_loss: 1.3851 | g_loss: 0.7968 Epoch [ 7/ 10] | d_loss: 1.3712 | g_loss: 0.8019 Epoch [ 7/ 10] | d_loss: 1.3765 | g_loss: 0.8221 Epoch [ 7/ 10] | d_loss: 1.3293 | g_loss: 0.8171 Epoch [ 7/ 10] | d_loss: 1.3969 | g_loss: 0.8406 Epoch [ 7/ 10] | d_loss: 1.3548 | g_loss: 0.8025 Epoch [ 7/ 10] | d_loss: 1.3454 | g_loss: 0.8380 Epoch [ 7/ 10] | d_loss: 1.3425 | g_loss: 0.8348 Epoch [ 7/ 10] | d_loss: 1.3291 | g_loss: 0.8420 Epoch [ 7/ 10] | d_loss: 1.3423 | g_loss: 0.8239 Epoch [ 7/ 10] | d_loss: 1.3457 | g_loss: 0.8543 Epoch [ 7/ 10] | d_loss: 1.3833 | g_loss: 0.8482 Epoch [ 7/ 10] | d_loss: 1.3510 | g_loss: 0.7973 Epoch [ 7/ 10] | d_loss: 1.3922 | g_loss: 0.8619 Epoch [ 7/ 10] | d_loss: 1.3518 | g_loss: 0.8376 Epoch [ 7/ 10] | d_loss: 1.3076 | g_loss: 0.8619 Epoch [ 7/ 10] | d_loss: 1.3040 | g_loss: 0.8441 Epoch [ 7/ 10] | d_loss: 1.3219 | g_loss: 0.8215 Epoch [ 7/ 10] | d_loss: 1.3494 | g_loss: 0.8055 Epoch [ 7/ 10] | d_loss: 1.3423 | g_loss: 0.8435 Epoch [ 7/ 10] | d_loss: 1.3590 | g_loss: 0.8167 Epoch [ 7/ 10] | d_loss: 1.3572 | g_loss: 0.8338 Epoch [ 7/ 10] | d_loss: 1.3633 | g_loss: 0.8664 Epoch [ 7/ 10] | d_loss: 1.3414 | g_loss: 0.8292 Epoch [ 7/ 10] | d_loss: 1.3249 | g_loss: 0.8556 Epoch [ 7/ 10] | d_loss: 1.3930 | g_loss: 0.8822 Epoch [ 7/ 10] | d_loss: 1.3841 | g_loss: 0.7859 Epoch [ 7/ 10] | d_loss: 1.3865 | g_loss: 0.7863 Epoch [ 7/ 10] | d_loss: 1.3755 | g_loss: 0.7994 Epoch [ 7/ 10] | d_loss: 1.3737 | g_loss: 0.8082 Epoch [ 7/ 10] | d_loss: 1.3758 | g_loss: 0.8040 Epoch [ 7/ 10] | d_loss: 1.3705 | g_loss: 0.8117 Epoch [ 7/ 10] | d_loss: 1.3696 | g_loss: 0.8033 Epoch [ 7/ 10] | d_loss: 1.3774 | g_loss: 0.8006 Epoch [ 7/ 10] | d_loss: 1.3682 | g_loss: 0.8239 Epoch [ 7/ 10] | d_loss: 1.3672 | g_loss: 0.8007 Epoch [ 7/ 10] | d_loss: 1.3582 | g_loss: 0.8389 Epoch [ 7/ 10] | d_loss: 1.2932 | g_loss: 0.8593 Epoch [ 7/ 10] | d_loss: 1.3203 | g_loss: 0.8086 Epoch [ 7/ 10] | d_loss: 1.3218 | g_loss: 0.8448 Epoch [ 7/ 10] | d_loss: 1.3359 | g_loss: 0.8676 Epoch [ 7/ 10] | d_loss: 1.3155 | g_loss: 0.9000 Epoch [ 7/ 10] | d_loss: 1.3545 | g_loss: 0.8599 Epoch [ 8/ 10] | d_loss: 1.3467 | g_loss: 0.8630 Epoch [ 8/ 10] | d_loss: 1.2731 | g_loss: 0.7910 Epoch [ 8/ 10] | d_loss: 1.3795 | g_loss: 0.8040 Epoch [ 8/ 10] | d_loss: 1.3889 | g_loss: 0.7889 Epoch [ 8/ 10] | d_loss: 1.3785 | g_loss: 0.8205 Epoch [ 8/ 10] | d_loss: 1.3875 | g_loss: 0.7807 Epoch [ 8/ 10] | d_loss: 1.3688 | g_loss: 0.7893 Epoch [ 8/ 10] | d_loss: 1.3741 | g_loss: 0.8088 Epoch [ 8/ 10] | d_loss: 1.3644 | g_loss: 0.8549 Epoch [ 8/ 10] | d_loss: 1.3602 | g_loss: 0.8260 Epoch [ 8/ 10] | d_loss: 1.3193 | g_loss: 0.8206 Epoch [ 8/ 10] | d_loss: 1.3367 | g_loss: 0.8560 Epoch [ 8/ 10] | d_loss: 1.3908 | g_loss: 0.8112 Epoch [ 8/ 10] | d_loss: 1.3357 | g_loss: 0.8610 Epoch [ 8/ 10] | d_loss: 1.3467 | g_loss: 0.8210 Epoch [ 8/ 10] | d_loss: 1.3120 | g_loss: 0.8675 Epoch [ 8/ 10] | d_loss: 1.3720 | g_loss: 0.8765 Epoch [ 8/ 10] | d_loss: 1.3860 | g_loss: 0.8872 Epoch [ 8/ 10] | d_loss: 1.4085 | g_loss: 0.8433 Epoch [ 8/ 10] | d_loss: 1.3890 | g_loss: 0.8164 Epoch [ 8/ 10] | d_loss: 1.3460 | g_loss: 0.8504 Epoch [ 8/ 10] | d_loss: 1.3487 | g_loss: 0.7894 Epoch [ 8/ 10] | d_loss: 1.3748 | g_loss: 0.7983 Epoch [ 8/ 10] | d_loss: 1.3700 | g_loss: 0.8647 Epoch [ 8/ 10] | d_loss: 1.3780 | g_loss: 0.8045 Epoch [ 8/ 10] | d_loss: 1.3438 | g_loss: 0.8306 Epoch [ 8/ 10] | d_loss: 1.3748 | g_loss: 0.8405 Epoch [ 8/ 10] | d_loss: 1.3524 | g_loss: 0.8086 Epoch [ 8/ 10] | d_loss: 1.3398 | g_loss: 0.8421 Epoch [ 8/ 10] | d_loss: 1.3413 | g_loss: 0.8779 Epoch [ 8/ 10] | d_loss: 1.3333 | g_loss: 0.8101 Epoch [ 8/ 10] | d_loss: 1.3003 | g_loss: 0.8552 Epoch [ 8/ 10] | d_loss: 1.3216 | g_loss: 0.8580 Epoch [ 8/ 10] | d_loss: 1.2918 | g_loss: 0.8444 Epoch [ 8/ 10] | d_loss: 1.3879 | g_loss: 0.8148 Epoch [ 8/ 10] | d_loss: 1.3606 | g_loss: 0.8449 Epoch [ 8/ 10] | d_loss: 1.3924 | g_loss: 0.8206 Epoch [ 8/ 10] | d_loss: 1.3842 | g_loss: 0.7898 Epoch [ 8/ 10] | d_loss: 1.3788 | g_loss: 0.7960 Epoch [ 8/ 10] | d_loss: 1.3825 | g_loss: 0.7971 Epoch [ 8/ 10] | d_loss: 1.3719 | g_loss: 0.8011 Epoch [ 8/ 10] | d_loss: 1.3729 | g_loss: 0.8101 Epoch [ 8/ 10] | d_loss: 1.3750 | g_loss: 0.8018 Epoch [ 8/ 10] | d_loss: 1.3607 | g_loss: 0.7828 Epoch [ 8/ 10] | d_loss: 1.3568 | g_loss: 0.8048 Epoch [ 8/ 10] | d_loss: 1.3592 | g_loss: 0.8251 Epoch [ 8/ 10] | d_loss: 1.3163 | g_loss: 0.8553 Epoch [ 8/ 10] | d_loss: 1.3320 | g_loss: 0.8460 Epoch [ 8/ 10] | d_loss: 1.3692 | g_loss: 0.8155 Epoch [ 8/ 10] | d_loss: 1.3712 | g_loss: 0.8276 Epoch [ 8/ 10] | d_loss: 1.3502 | g_loss: 0.8567 Epoch [ 8/ 10] | d_loss: 1.3462 | g_loss: 0.8764 Epoch [ 8/ 10] | d_loss: 1.3228 | g_loss: 0.8670 Epoch [ 8/ 10] | d_loss: 1.3443 | g_loss: 0.8638 Epoch [ 8/ 10] | d_loss: 1.3809 | g_loss: 0.8084 Epoch [ 8/ 10] | d_loss: 1.3750 | g_loss: 0.8038 Epoch [ 8/ 10] | d_loss: 1.3528 | g_loss: 0.8632 Epoch [ 9/ 10] | d_loss: 1.3396 | g_loss: 0.8541 Epoch [ 9/ 10] | d_loss: 1.3816 | g_loss: 0.8664 Epoch [ 9/ 10] | d_loss: 1.3364 | g_loss: 0.8426 Epoch [ 9/ 10] | d_loss: 1.3296 | g_loss: 0.8375 Epoch [ 9/ 10] | d_loss: 1.3532 | g_loss: 0.8551 Epoch [ 9/ 10] | d_loss: 1.2924 | g_loss: 0.6955 Epoch [ 9/ 10] | d_loss: 1.3408 | g_loss: 0.8318 Epoch [ 9/ 10] | d_loss: 1.3871 | g_loss: 0.8280 Epoch [ 9/ 10] | d_loss: 1.3361 | g_loss: 0.8564 Epoch [ 9/ 10] | d_loss: 1.2970 | g_loss: 0.8536 Epoch [ 9/ 10] | d_loss: 1.3772 | g_loss: 0.8196 Epoch [ 9/ 10] | d_loss: 1.3753 | g_loss: 0.8398 Epoch [ 9/ 10] | d_loss: 1.3419 | g_loss: 0.7727 Epoch [ 9/ 10] | d_loss: 1.3877 | g_loss: 0.8391 Epoch [ 9/ 10] | d_loss: 1.3617 | g_loss: 0.8185 Epoch [ 9/ 10] | d_loss: 1.3373 | g_loss: 0.8428 Epoch [ 9/ 10] | d_loss: 1.3767 | g_loss: 0.8572 Epoch [ 9/ 10] | d_loss: 1.3811 | g_loss: 0.7978 Epoch [ 9/ 10] | d_loss: 1.3853 | g_loss: 0.8472 Epoch [ 9/ 10] | d_loss: 1.3882 | g_loss: 0.7896 Epoch [ 9/ 10] | d_loss: 1.3725 | g_loss: 0.8132 Epoch [ 9/ 10] | d_loss: 1.3492 | g_loss: 0.8308 Epoch [ 9/ 10] | d_loss: 1.3371 | g_loss: 0.8132 Epoch [ 9/ 10] | d_loss: 1.3464 | g_loss: 0.8346 Epoch [ 9/ 10] | d_loss: 1.3625 | g_loss: 0.8196 Epoch [ 9/ 10] | d_loss: 1.3561 | g_loss: 0.8610 Epoch [ 9/ 10] | d_loss: 1.3716 | g_loss: 0.8280 Epoch [ 9/ 10] | d_loss: 1.3148 | g_loss: 0.8443 Epoch [ 9/ 10] | d_loss: 1.3255 | g_loss: 0.8539 Epoch [ 9/ 10] | d_loss: 1.3224 | g_loss: 0.8753 Epoch [ 9/ 10] | d_loss: 1.3674 | g_loss: 0.8464 Epoch [ 9/ 10] | d_loss: 1.3136 | g_loss: 0.9073 Epoch [ 9/ 10] | d_loss: 1.3510 | g_loss: 0.8304 Epoch [ 9/ 10] | d_loss: 1.3656 | g_loss: 0.8918 Epoch [ 9/ 10] | d_loss: 1.3989 | g_loss: 0.8494 Epoch [ 9/ 10] | d_loss: 1.4021 | g_loss: 0.7996 Epoch [ 9/ 10] | d_loss: 1.3584 | g_loss: 0.7933 Epoch [ 9/ 10] | d_loss: 1.3592 | g_loss: 0.8536 Epoch [ 9/ 10] | d_loss: 1.3242 | g_loss: 0.8689 Epoch [ 9/ 10] | d_loss: 1.3175 | g_loss: 0.8409 Epoch [ 9/ 10] | d_loss: 1.3344 | g_loss: 0.8490 Epoch [ 9/ 10] | d_loss: 1.3353 | g_loss: 0.8300 Epoch [ 9/ 10] | d_loss: 1.3030 | g_loss: 0.8622 Epoch [ 9/ 10] | d_loss: 1.3565 | g_loss: 0.8595 Epoch [ 9/ 10] | d_loss: 1.3189 | g_loss: 0.8291 Epoch [ 9/ 10] | d_loss: 1.3703 | g_loss: 0.9218 Epoch [ 9/ 10] | d_loss: 1.3438 | g_loss: 0.8402 Epoch [ 9/ 10] | d_loss: 1.3620 | g_loss: 0.8509 Epoch [ 9/ 10] | d_loss: 1.3333 | g_loss: 0.8515 Epoch [ 9/ 10] | d_loss: 1.3876 | g_loss: 0.8232 Epoch [ 9/ 10] | d_loss: 1.3801 | g_loss: 0.8145 Epoch [ 9/ 10] | d_loss: 1.3896 | g_loss: 0.7928 Epoch [ 9/ 10] | d_loss: 1.3775 | g_loss: 0.7981 Epoch [ 9/ 10] | d_loss: 1.3855 | g_loss: 0.7924 Epoch [ 9/ 10] | d_loss: 1.3891 | g_loss: 0.7990 Epoch [ 9/ 10] | d_loss: 1.3775 | g_loss: 0.8286 Epoch [ 9/ 10] | d_loss: 1.3741 | g_loss: 0.8287 Epoch [ 10/ 10] | d_loss: 1.3604 | g_loss: 0.8070 Epoch [ 10/ 10] | d_loss: 1.3788 | g_loss: 0.8017 Epoch [ 10/ 10] | d_loss: 1.3647 | g_loss: 0.8185 Epoch [ 10/ 10] | d_loss: 1.3553 | g_loss: 0.8040 Epoch [ 10/ 10] | d_loss: 1.3613 | g_loss: 0.8039 Epoch [ 10/ 10] | d_loss: 1.3671 | g_loss: 0.8125 Epoch [ 10/ 10] | d_loss: 1.3668 | g_loss: 0.8533 Epoch [ 10/ 10] | d_loss: 1.3563 | g_loss: 0.8160 Epoch [ 10/ 10] | d_loss: 1.3517 | g_loss: 0.8423 Epoch [ 10/ 10] | d_loss: 1.3023 | g_loss: 0.8799 Epoch [ 10/ 10] | d_loss: 1.3927 | g_loss: 0.9026 Epoch [ 10/ 10] | d_loss: 1.3624 | g_loss: 0.8073 Epoch [ 10/ 10] | d_loss: 1.3818 | g_loss: 0.8493 Epoch [ 10/ 10] | d_loss: 1.3656 | g_loss: 0.8050 Epoch [ 10/ 10] | d_loss: 1.3880 | g_loss: 0.8095 Epoch [ 10/ 10] | d_loss: 1.3739 | g_loss: 0.8076 Epoch [ 10/ 10] | d_loss: 1.3665 | g_loss: 0.7989 Epoch [ 10/ 10] | d_loss: 1.3734 | g_loss: 0.7947 Epoch [ 10/ 10] | d_loss: 1.3674 | g_loss: 0.7954 Epoch [ 10/ 10] | d_loss: 1.3688 | g_loss: 0.7916 Epoch [ 10/ 10] | d_loss: 1.3828 | g_loss: 0.7932 Epoch [ 10/ 10] | d_loss: 1.3657 | g_loss: 0.7940 Epoch [ 10/ 10] | d_loss: 1.3654 | g_loss: 0.7998 Epoch [ 10/ 10] | d_loss: 1.3714 | g_loss: 0.7878 Epoch [ 10/ 10] | d_loss: 1.3620 | g_loss: 0.7873 Epoch [ 10/ 10] | d_loss: 1.3826 | g_loss: 0.8003 Epoch [ 10/ 10] | d_loss: 1.3675 | g_loss: 0.8156 Epoch [ 10/ 10] | d_loss: 1.3741 | g_loss: 0.8622 Epoch [ 10/ 10] | d_loss: 1.3564 | g_loss: 0.8377 Epoch [ 10/ 10] | d_loss: 1.3510 | g_loss: 0.8286 Epoch [ 10/ 10] | d_loss: 1.3251 | g_loss: 0.8636 Epoch [ 10/ 10] | d_loss: 1.2989 | g_loss: 0.8453 Epoch [ 10/ 10] | d_loss: 1.3145 | g_loss: 0.8124 Epoch [ 10/ 10] | d_loss: 1.3722 | g_loss: 0.8257 Epoch [ 10/ 10] | d_loss: 1.3657 | g_loss: 0.7977 Epoch [ 10/ 10] | d_loss: 1.3834 | g_loss: 0.8130 Epoch [ 10/ 10] | d_loss: 1.3597 | g_loss: 0.8098 Epoch [ 10/ 10] | d_loss: 1.3490 | g_loss: 0.8081 Epoch [ 10/ 10] | d_loss: 1.3484 | g_loss: 0.8116 Epoch [ 10/ 10] | d_loss: 1.3518 | g_loss: 0.8060 Epoch [ 10/ 10] | d_loss: 1.3601 | g_loss: 0.8514 Epoch [ 10/ 10] | d_loss: 1.3550 | g_loss: 0.8159 Epoch [ 10/ 10] | d_loss: 1.3565 | g_loss: 0.8582 Epoch [ 10/ 10] | d_loss: 1.2981 | g_loss: 0.8632 Epoch [ 10/ 10] | d_loss: 1.3087 | g_loss: 0.8558 Epoch [ 10/ 10] | d_loss: 1.3597 | g_loss: 0.8044 Epoch [ 10/ 10] | d_loss: 1.3525 | g_loss: 0.8284 Epoch [ 10/ 10] | d_loss: 1.3357 | g_loss: 0.8713 Epoch [ 10/ 10] | d_loss: 1.3375 | g_loss: 0.8558 Epoch [ 10/ 10] | d_loss: 1.2997 | g_loss: 0.9120 Epoch [ 10/ 10] | d_loss: 1.3373 | g_loss: 0.8673 Epoch [ 10/ 10] | d_loss: 1.3782 | g_loss: 0.9199 Epoch [ 10/ 10] | d_loss: 1.3574 | g_loss: 0.8255 Epoch [ 10/ 10] | d_loss: 1.3240 | g_loss: 0.8602 Epoch [ 10/ 10] | d_loss: 1.3721 | g_loss: 0.8386 Epoch [ 10/ 10] | d_loss: 1.3726 | g_loss: 0.8611 Epoch [ 10/ 10] | d_loss: 1.3789 | g_loss: 0.8161 Training time: 1704.8 seconds
Plot the training losses for the generator and discriminator, recorded after each epoch.
fig, ax = plt.subplots()
losses = np.array(losses)
plt.plot(losses.T[0], label='Discriminator', alpha=0.5)
plt.plot(losses.T[1], label='Generator', alpha=0.5)
plt.title("Training Losses")
plt.legend()
<matplotlib.legend.Legend at 0x1c20d87e850>
View samples of images from the generator, and answer a question about the strengths and weaknesses of your trained models.
# helper function for viewing a list of passed in sample images
def view_samples(epoch, samples):
fig, axes = plt.subplots(figsize=(16,4), nrows=2, ncols=8, sharey=True, sharex=True)
for ax, img in zip(axes.flatten(), samples[epoch]):
img = img.detach().cpu().numpy()
img = np.transpose(img, (1, 2, 0))
img = ((img + 1)*255 / (2)).astype(np.uint8)
ax.xaxis.set_visible(False)
ax.yaxis.set_visible(False)
im = ax.imshow(img.reshape((32,32,3)))
# Load samples from generator, taken while training
with open('train_samples.pkl', 'rb') as f:
samples = pkl.load(f)
# epoch=1, lr=0.001, dim=512, z=128, batch=32
#_ = view_samples(-1, samples)
# epoch=1, lr=0.001, dim=128, z=128, batch=32
#_ = view_samples(-1, samples)
# epoch=10, lr=0.001, dim=128, z=128, batch=32 t=1710 seconds
#_ = view_samples(-1, samples)
# epoch=50, lr=0.001, dim=128, z=128, batch=32, training time 1.5h
#_ = view_samples(-1, samples)
# decreasing learning rate
# epoch=10, lr=0.0002, dim=128, z=128, batch=32 t=1713 seconds
#_ = view_samples(-1, samples)
# decreasing learning rate, increase number of epochs
# epoch=50, lr=0.0002, dim=128, z=128, batch=32
#_ = view_samples(-1, samples)
# dropout after each convolutional layer in discriminator
# epoch=10, lr=0.0002, dim=128, z=128, batch=32 t=1715 seconds
#_ = view_samples(-1, samples)
# no dropout after each convolutional layer in discriminator
# epoch=10, lr=0.0002, dim=128, z=128, batch=32 t=1715 seconds
#_ = view_samples(-1, samples)
# decreasing learning rate
# epoch=10, lr=0.0002, dim=128, z=128, batch=32 t=1703 seconds
#_ = view_samples(-1, samples)
# epoch=50, lr=0.0002, dim=128, z=128, batch=32 t=8531 seconds
#_ = view_samples(-1, samples)
# epoch=10, lr=0.0002, dim=128, z=128, batch=32 t=8531 seconds
#_ = view_samples(-1, samples)
# epoch=10, lr=0.0002, dim=128, z=128, batch=32 t=8531 seconds
#_ = view_samples(-1, samples)
When you answer this question, consider the following factors:
Answer: (Write your answer in this cell) Data set is obviously biased and represents a unique subset of human faces. Apparently, this limitation allows to have relatively shallow generator and discriminator which have only three convolutional layers each. Increasing number of layers to capture more image features or dimension of layers to increase number of "filters" didn't result in a significant improvement of generated images. Changing learning rate from 0.001 to 0.0002 had a very significant impact on the output. Image quality at learning rate of 0.001 and training at 50 epoches was about the same as that images trained at 0.0002 learning rate and just 10 epochs - about 1/5 of training time of 50 epoches. Increasing number of epochs from 10 to 50 was decreasing discriminator loss and images were getting better. Interestingly, this increase was not leading to significant improvement. For example, [45] was trained on 10 epochs and [28] on 50. Another example of [43] and [88] on 10 epochs vs. [78] on 50. Model behaviour on 10 epochs was not stable. For example, [45] vs [73]. Apparently, increasing number of epochs, although very slowly, decreases loss whereas on smaller number of epochs loss may stay at slightly higher or slightly lower constant level. Likely improvement for this particular architecure wouldbe increase of number of epochs.
When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_face_generation.ipynb" and save it as a HTML file under "File" -> "Download as". Include the "problem_unittests.py" files in your submission.